Skip to content

Conversation

@sarutak
Copy link
Member

@sarutak sarutak commented Nov 6, 2014

Httpbroadcast sets read timeout but doesn't set connection timeout.

@SparkQA
Copy link

SparkQA commented Nov 6, 2014

Test build #22964 has finished for PR 3122 at commit dce0f5a.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • public final class JavaGradientBoostedTrees
    • case class Params(
    • public final class LimitedInputStream extends FilterInputStream
    • public class ShuffleSecretManager implements SecretKeyHolder
    • public class YarnShuffleService extends AuxiliaryService
    • public class HadoopConfigProvider extends ConfigProvider
    • case class AllocatedBlocks(streamIdToAllocatedBlocks: Map[Int, Seq[ReceivedBlockInfo]])
    • class ReceiverTracker(ssc: StreamingContext, skipReceiverLaunch: Boolean = false) extends Logging

@sarutak sarutak force-pushed the httpbroadcast-timeout branch from dce0f5a to c7f3a56 Compare November 6, 2014 00:35
@SparkQA
Copy link

SparkQA commented Nov 6, 2014

Test build #22967 has finished for PR 3122 at commit c7f3a56.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@squito
Copy link
Contributor

squito commented Nov 9, 2014

This looks good, but could also explain what necessitates this change? Did you observe some error? If nothing else, just putting the error you observed in the JIRA would help somebody else find this patch if they run into the error as well.

@sarutak
Copy link
Member Author

sarutak commented Nov 10, 2014

Actually I don't hit any issues caused by not setting connection timeout but the host trying to connect to a host wait for establishing forever when server side is hang.
In fact, org.apache.spark.util.Utils#doFetchFile, URLConnection is set timeout.

@rxin
Copy link
Contributor

rxin commented Nov 15, 2014

Ok this makes sense. I'm merging in master & branch-1.2.

@rxin
Copy link
Contributor

rxin commented Nov 15, 2014

Thanks a lot!

@asfgit asfgit closed this in 60969b0 Nov 15, 2014
asfgit pushed a commit that referenced this pull request Nov 15, 2014
Httpbroadcast sets read timeout but doesn't set connection timeout.

Author: Kousuke Saruta <[email protected]>

Closes #3122 from sarutak/httpbroadcast-timeout and squashes the following commits:

c7f3a56 [Kousuke Saruta] Added Connection timeout for Http Connection to HttpBroadcast.scala

(cherry picked from commit 60969b0)
Signed-off-by: Reynold Xin <[email protected]>
@sarutak sarutak deleted the httpbroadcast-timeout branch April 11, 2015 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants